173 research outputs found
Kernel-based Information Criterion
This paper introduces Kernel-based Information Criterion (KIC) for model
selection in regression analysis. The novel kernel-based complexity measure in
KIC efficiently computes the interdependency between parameters of the model
using a variable-wise variance and yields selection of better, more robust
regressors. Experimental results show superior performance on both simulated
and real data sets compared to Leave-One-Out Cross-Validation (LOOCV),
kernel-based Information Complexity (ICOMP), and maximum log of marginal
likelihood in Gaussian Process Regression (GPR).Comment: We modified the reference 17, and the subcaptions of Figure
Metric State Space Reinforcement Learning for a Vision-Capable Mobile Robot
We address the problem of autonomously learning controllers for
vision-capable mobile robots. We extend McCallum's (1995) Nearest-Sequence
Memory algorithm to allow for general metrics over state-action trajectories.
We demonstrate the feasibility of our approach by successfully running our
algorithm on a real mobile robot. The algorithm is novel and unique in that it
(a) explores the environment and learns directly on a mobile robot without
using a hand-made computer model as an intermediate step, (b) does not require
manual discretization of the sensor input space, (c) works in piecewise
continuous perceptual spaces, and (d) copes with partial observability.
Together this allows learning from much less experience compared to previous
methods.Comment: 14 pages, 8 figure
Evolino for recurrent support vector machines
Traditional Support Vector Machines (SVMs) need pre-wired finite time windows
to predict and classify time series. They do not have an internal state
necessary to deal with sequences involving arbitrary long-term dependencies.
Here we introduce a new class of recurrent, truly sequential SVM-like devices
with internal adaptive states, trained by a novel method called EVOlution of
systems with KErnel-based outputs (Evoke), an instance of the recent Evolino
class of methods. Evoke evolves recurrent neural networks to detect and
represent temporal dependencies while using quadratic programming/support
vector regression to produce precise outputs. Evoke is the first SVM-based
mechanism learning to classify a context-sensitive language. It also
outperforms recent state-of-the-art gradient-based recurrent neural networks
(RNNs) on various time series prediction tasks.Comment: 10 pages, 2 figure
NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations
This paper introduces Non-Autonomous Input-Output Stable Network (NAIS-Net),
a very deep architecture where each stacked processing block is derived from a
time-invariant non-autonomous dynamical system. Non-autonomy is implemented by
skip connections from the block input to each of the unrolled processing stages
and allows stability to be enforced so that blocks can be unrolled adaptively
to a pattern-dependent processing depth. NAIS-Net induces non-trivial,
Lipschitz input-output maps, even for an infinite unroll length. We prove that
the network is globally asymptotically stable so that for every initial
condition there is exactly one input-dependent equilibrium assuming tanh units,
and multiple stable equilibria for ReL units. An efficient implementation that
enforces the stability under derived conditions for both fully-connected and
convolutional layers is also presented. Experimental results show how NAIS-Net
exhibits stability in practice, yielding a significant reduction in
generalization gap compared to ResNets.Comment: NIPS 201
- …